Technology's New Fields of Dreams in Dance

Technology and dance have long been ardent bedfellows. No other theatrical art gobbles up illusions and tricks quite as greedily and spits them out quite as intriguingly altered. Gaslight was a new technology without which the romantic ballets Giselle and La Sylphide could not have existed. Without electric light such exotic adventures in sunshine as Le Corsaire or Don Quixote could not have partied over the late 19th-century St Petersburg stage.

In the 20th century, hand-drawn film animation allowed animals to dance and speak, and Merce Cunningham seized on computer software to explore motion capture, making supra-natural choices in manipulating human movement.

Digital video and audio offer yet more possibilities: Akram Khan’s DESH and Russell Maliphant’s AfterLight are only two recent contemporary dance productions that used the space-altering potential of digital light and sound to take us in an instant into fiction or dreamland.

But when does technology override good old-fashioned scenery magic? Ropes, flapping material and puppetry retain a great hold on our childish memories, and we won't easily give them up for the flashy ephemeralities of IT, unless the technology is truly magical. For in dance we're uniquely tolerant of rough edges - we're more interested in being removed to places that theatre and opera can't go.

Crystal Ballet wants to put top ballet dancers onto your mobile and into your pocket

As a result, we'd better get used to the fact that creative artists in the dance world are seizing on technology to make new ways of telling their stories, new ways of looking at dance and being affected by it.

Next week a new enterprise called Crystal Ballet launches, a British project to put top ballet dancers onto your mobile and into your pocket. A mouthwatering list of Royal Ballet and English National Ballet stars and choreographers are involved: Alina Cojocaru, Johan Kobborg, Sarah Lamb, Steven McRae, Daria Klimentová and Vadim Muntagirov among those who will be downloadable onto your Galaxy or iPhone in choreography created specially for mobile viewing by Kim Brandstrup and Ernst Meisner.

Crystal Ballet’s purpose looks to be more about opening access, rather than messing with the art - in a sense, to create ballet you can carry around on the tube or bus, downloaded alongside your apps. The traditional triggers of quality and star appeal remain there. The picture size requires a distillation to essences of music, movement, human nuance.

The first of what’s expected to be many mobile ballets, called Genesis (appropriately), is scheduled to go on sale for £15 from this Sunday, when Crystal Ballet's official launch takes place in Japan - convenient for the Royal Ballet, whose personnel will be on tour there. The big European launch takes place in September in London, and by then it will be starting to become clear whether a young, app-mad public will buy this new vehicle for art, download ballet and fall in love with it (see the trailer below).

3D goes the opposite way - it's not miniaturising, it's expanding. Already movie-goers have fallen in love with 3D in a big way, which has moved over from educational tool to mainstream medium, thanks to Avatar and, recently, Life of Pi. It’s proving a tougher nut to crack for dance.

Not least, because it’s hair-raisingly labour-intensive and cumbersome for 3D filming of a live performance to capture the mercurial way that dancers move and occupy space, to "take us there". That's the first obvious use; the other is to use 3D as a creative tool in itself inside the theatre itself, to offer a magical new palette to invent theatre ideas with, just as gaslight opened up the possibility of nocturnal ballet scenes and ghostly illusions.

There’s been an example of each of these in this past fortnight: one, the live 3D relay from the Mariinsky Ballet in St Petersburg of Swan Lake, the other, an experimental new production in Paris of a contemporary dance spectacle collaborating with 3D imaging techniques. The live relay attempts to break down the artificiality of the screen and take you into the live theatre somewhere else in the world - the other, conversely, attempted to transfigure the experience inside the theatre itself with visual tricks.

The iconic St Petersburg company, known worldwide for its classical purity, had already made a dud go at 3D filming with Giselle two years ago (with Natalia Osipova and Leonid Sarafanov in the leads); its Swan Lake live 3D relay a fortnight ago was a much superior experience. It had called on James Cameron, director of Avatar and Titanic 3D film productions, and an experienced dance-film director, Ross McGibbon, who had cut his 3D teeth on a fine rendition of Matthew Bourne’s modern Swan Lake last year.

The expertise showed - the cartoon layers of the Giselle were gone; here were more intuitive camera points, so that the swangirls flocked with a real sense of depth from far to near.

The Mariinsky’s aim is to replicate the live experience inside its theatre for a global audience. But was 3D so much better than 2D in achieving that? I don’t think so, on that occasion, though not through fault of the technology. Strangely enough, what let it down was the central performances.

Filming, whatever the dimension, exposes the uncommitted, and Ekaterina Kondaurova, as tall and willowy as could be desired, was not an imaginative enough storyteller as Odette/Odile to take us there. Her Siegfried too, Timur Askerov, performed like a handsome pupil, unaffected by his story - a lack of belief that again is magnified by this technology. For the more reality you are invited to experience visually, the more real you expect to feel the story inside it.

The Mariinsky Swan Lake felt more about showing a company's trademark stylisation and visual brand - the world's most popular ballet performed by the world's most iconic company - than a decision to bring its most expressive and rewarding performers to cast a magic imaginative spell. Performers must realise how they need to rework their stage habits afresh to push the fantasy down the wire and inject us with its addictive poignancy. That way, we'll come back for more.

In the end, in 3D live relaying as with Crystal Ballet's pocket dancing, it’s going to be old-fashioned things that count - the commitment of the performance, the quality of the scenery - which will decide whether mass audiences will flock to buy into the new technological dimension.

3D digital visuals can transform a bare white box of a stage into... well, anything you want. Absolutely anything

It’s rather more obvious that 3D digital art visuals have a big potential to captivate audiences in the live theatre, as a premiere in Paris last week demonstrated. The French star ballerina Marie-Claude Pietragalla, once one of Paris Opera Ballet’s decorations, and later the controversial director of the Ballet de Marseille, has launched a brave new enterprise to use 3D digital visuals to transform a bare white box of a stage into... well, anything you want. And that means absolutely, but absolutely anything.

In their production Mr and Mrs Dream (M et Mme Rêve), she and her handsome partner, Jean Derouault, beamed in a trice from a splendid 19th-century salon to galaxies in outer space to fields full of marching rhinoceroses, all with the brush of a finger on a trackpad.

The storyboard was written in collaboration with a digital technology company specialising in 3D "experience". It was based on the playwright of the absurd, Eugene Ionesco, who was made the legal excuse for a host of wildly disjointed imagery, but the effect was of a trade demonstration of the multitude of possibilities available with 3D digital “experience” software.

What was undoubtedly fascinating the invited audience in Paris’s Halle Freyssinet - mostly industry professionals and journalists - was to see the nursery steps of a gorgeous new way to make worlds happen, to make fantasy happen. The choreography and presence of the two well-known and glamorous dancers were not, on this showing, able to make any very interesting dance-theatre, but you have to allow for the early stages of striking a balance with so imposing and charismatic a new stage technique.

The point about 3D visual “scenery” is that it is as ephemeral and malleable and surreally instant in response as if you were a witch waving a magic wand. The dancers in Mr and Mrs Dream hopscotched over meteors floating in outer space, or broke up into multiple copies of themselves. They jumped into visual puddles that sploshed in slow-motion, they swept up their arm and made rainbows of letters fly over their heads (pictured below). The potential is there to improvise the effects live.

Derouault had used good old motion capture to create an impressive final army of cartoon rhinoceros soldiers marching to Wagner’s over-familiar Ride of the Valkyries. And there’s the rub. Artistically, the production was too strewn with clichés, too undercooked in choreography, and too tediously long and vacuous in places, to be a real pleasure to experience.

No matter how exciting the technology, the quality of the artistry will always be more important, which includes the visual art direction - surely this is going to give a new world of opportunity to visual and graphic artists.

Another valuable potential is that the technology's commercial development in modelling buildings offers an exciting way to travel backwards, to decorate timeless classics as well as visualising contemporary journeys into psyche and dream. The one problem is that you can’t sit (yet) on a 3D throne - so, as yet, when someone does Swan Lake this way (and I hope they will), there will still have to be somewhere for the Queen to sit.  

Next page: how the 3D theatre software works

How 3D theatre software developed

The 3D digital “experience” used by the M et Mme Rêve production has been evolved from 3D modelling software created by Dassault Systèmes to design industrial products.

It’s worth tracking their industrial previous form in order to understand the potential for theatre. 3D computer applications that had originally been developed in the early 1980s to build and test aeroplanes for the French government. Alternate components could be digitally modelled in different shapes and materials, their properties evaluated, and design decisions made in a more sophisticated way that could actively test speculation, potential usage and arising risk factors.

This was the key for the move into the cultural sphere - speculation became now not a potential danger, but an active benefit in design.

As the scientists' technical engineering expertise became increasingly nuanced, they moved into partnership with academics and researchers, attempting to find the reality in mythical and legendary concepts and buildings. Dassault Systèmes’s director of interactive systems Mehdi Tayoubi says the cultural spin-off was a deliberate step in order to improve the effectiveness and human interactivity of the applications, speeding up their development and discovering new potential.

In 2005 Dassault were asked by a French architect to help him test his theory about how the Cheops pyramid at Giza was constructed 4,500 years ago. A life-sized virtual simulation was created by Dassault, factoring in known research, and generated spectacular TV programmes (on the BBC's Timewatch among others) which suggested new directions for research and exhibits by Egyptologists and archaeological museums.

Last year a virtual 3D model of Paris evolving from antiquity to today was put online for academics, schools and the generally interested to explore and learn from. (Right, the technicians recreating historical Paris in their "salle de réalité")

The remarkable virtue of the 3D modelling concept was how it enabled a virtual hands-on experience to be gradually created, becoming more and more detailed and sophisticated with the input of factors on solid materials, building logic and the human psychology of imaginative concepts, drawn from contemporary records, writings and even paintings.

This “soft” side - the imaginative input by people into the making of their environment - is what Dassault say working on a cultural project hugely improves and fine-tunes. A basic construction application may programme known facts about materials and methods very well, but is much less good at understanding the part that in reality is played in every kind of construction by impulse, flaw, creativity and intention.

Increasingly Dassault has moved into large-scale cultural modelling that out-Cecil B De Milles Cecil B De Mille and offers an arresting new way to study history. The past is now becoming recoverable into a virtual reality. Their spectacular historical Paris 3D experience was shown live to 15,000 people last autumn in central Paris, and Louis XIV’s lost flagship, La Lune, wrecked in 1664, has been recreated in virtual 3D enabling both real and "virtual" archaeologists to take what appears to be an accurate trip into a chapter of history.

Below: a trailer by Dassault about the Paris 3D staging explains all

Accuracy and historical authenticity aside - and there are those who think the technology is in danger of leading discovery, rather than the other way around - it was a short imaginative step into the apparently ephemeral world of entertainment. Not least because the art direction of these 3D trips is increasingly becoming a major preoccupation for the scientists, rivalling the art direction in movie studios.

Last year the Dassault team collaborated with two comic strip artists to incorporate 3D into an animation film about the adventures of a steam train, François Schuiten’s La Douce. Along the way, the technicians built a virtual model of the original La Douce, a 1930s engine that was the fastest of its time, thus serving both art and historical science.

Working with two dancers might seem simplicity itself after modelling lost cities, but it's a much more delicate challenge

Although working with a mere two dancers might seem simplicity itself after dealing with peeling back legends and modelling lost cities, in fact it's a much more delicate and unruly challenge. Putting 3D technology live into a theatre sets the task of adapting to human movement and thought, rather than with solid objects, and to creative rather than logical maps.

People do not move in exactly programmable ways, and they don’t think in straight lines. Dancers, in particular, move exceptionally fast, and with purposeful gestures.

More than that, the technology becomes only as good as its artistic director. It’s no longer the unchallenged centre of attention - it’s a medium for someone else’s creativity. But that creator needs to understand the technology very well, to see its possibilities, to exploit it and wring out what this 3D experience offers that more orthodox stage techniques cannot. Where you get great technology being brought into the art world, you can't help expecting that great art be created with it - it's not enough just to ooh and aah over the gizmos.

The way it works

In film and audio recording it’s commonplace to record separate “tracks” and fit them together in the editing studio. Live theatre is a whole new ball-game for melding 3D digital visuals and warm bodies if they are genuinely to interact, rather than simply occur side-by-side.

The lab team at Dassault created a “magic box” consisting of light-grey screens, projectors and computers. The screens become the theatre’s three walls and floor; the six projectors play out the visuals throughout the 3D space. The apparatus dismantles and fits into a small lorry for easy transportation.

Above, the basic screen "magic box" with its computer sectors; below, a projection in M et Mme Rêve

The visuals are designed by art directors, and programmed into the specific 3DExperience software for the production, along with its soundtrack.

According to Dassault Systèmes’s Mehdi Tayoubi, about two-thirds of the sound and light elements were fixed, and another third contained possibilities for variability. While this is likely to be one place where the expected effects may break down, it’s also the place where creative theatre artists would most want to play and exercise their power.

How swiftly theatre artists adopt 3D stage imaging - as opposed to solid scenery and lighting artists - will not necessarily depend on being rich. Dassault are offering a licence for their software at an affordable 10,000 euros - the hardware is another matter, though one might imagine certain theatres investing in the screens, projectors and computers, as the artists tour with their own custom-made 3D productions all stored on a memory stick.

The way ahead

When art meets science, science always improves, and this is the mantra of the most forward-thinking technicians. In the sector of “techno-imaginary” industries - as defined by the French anthropologist Georges Balandier - the importance of fleshing out the skeleton of hard numbers with the flesh of human variability and interactivity has become a multibillion-dollar race.

Surely it won't be long before there's a Swan Lake there, created with a gripping new scenic artistry to revitalise dance's world of fairytale and fantasy

Apple got ahead of the giants IBM and Microsoft by realising this, sweeping away the assumed “need to know” and tech-speak, and redefining the personal computer market in idiot-proof, human-friendly terms by making instinct as important a directive as functional logic in building computer applications.

Artists, says Mehdi Tayoubi, give back to engineers a huge new set of challenges. By meeting and solving those more instinctive, sensory challenges, the better knowledge of interactivity can be applied straight back into industrial modelling and instructional software.

They acknowledge the debt of previous visionaries such as Walt Disney, whose animations were far from simply children’s entertainment - Mickey Mouse and Snow White launched an entire industry where imagination and engineering met, and gave birth to new ways of seeing what you imagined.

The key factor is going to be the takeover of the science by the artists. Without them, it's just a box of tricks. But the scientists do seem to be of the open-source mentality, wanting poor artists to exploit their toys while scientists feed off their by-product. And surely it won't be long before there's a Swan Lake there, with visual illusions far superior to the dreary scenery we're used to. Swans suddenly changed into swangirls, visuals to match Tchaikovsky, a gripping new scenic artistry to revitalise the world of fairytale and fantasy that is ballet.

• Dassault Systèmes' Mr and Mrs Dream website
• Mr and Mrs Dream is to be given a run of public performances in Paris's Grand Rex 12-22 March 2014
• Crystal Ballet website
  
• The Mariinsky Ballet's worldwide 3D screenings

A brief trailer about Mr and Mrs Dream with glimpses of the theatre effects

Find @ismeneb on Twitter